The present application relates to digital data processing, and more particularly to cache eviction for runtime systems.
A runtime system is a code execution environment that executes instructions or code in user requests and that provides runtime services for that code. Core runtime services can include functionality such as process, thread, and memory management (e.g., laying out entities in a server memory, managing references to entities, and garbage collecting entities). Enhanced runtime services can include functionality such as error handling and establishing security and connectivity.
One example of a runtime system is a virtual machine (VM). A VM is an abstract machine that can include an instruction set, a set of registers, a stack, a heap, and a method area, like a real machine or processor. A VM essentially acts as an interface between program code and the actual processor or hardware platform on which the program code is to be executed. The program code includes instructions from the VM instruction set that manipulate the resources of the VM. The VM executes instructions on the processor or hardware platform on which the VM is running, and manipulates the resources of that processor or hardware platform, so as to effect the instructions of the program code. In this way, the same program code can be executed on multiple processors or hardware platforms without having to be rewritten or re-compiled for each processor or hardware platform. Instead, a VM is implemented for each processor or hardware platform, and the same program code can be executed in each VM. The implementation of a VM can be in code that is recognized by the processor or hardware platform. Alternatively, the implementation of a VM can be in code that is built directly into a processor.
As an example, a Java source program can be compiled into program code known as bytecode. Bytecode can be executed on a Java VM running on any processor or platform. The Java VM can either interpret the bytecode one instruction at a time, or the bytecode can be further compiled for the real processor or platform using a just-in-time (JIT) compiler.
In addition to Java VMs, other examples of VMs include Advanced Business Application Programming language (ABAP) VMs and Common Language Runtime (CLR) VMs. ABAP is a programming language for developing applications for the SAP R/3 system, a widely installed business application system developed by SAP. The CLR is a managed code execution environment developed by Microsoft Corp. of Redmond, Wash.
When executing, runtime systems create and reference local data entities. Many different types of local entities can be created, including, for example, strings, constants, and variables, objects that are instances of a class, runtime representations of a class, and class loaders that are used to load class runtime representations.
When a local entity is no longer being used by a runtime system, the memory being used to store the local entity needs to be reclaimed—i.e., freed or otherwise released and returned to the pool of available memory—so that it can be used to store new data entities. In some environments, programmers must explicitly reclaim the memory they have allocated for local entities (e.g., by explicitly calling functions to free specified sections of memory). In other environments, the reclamation of memory is handled through a technique known as garbage collection.
Garbage collection is a process designed to identify and reclaim blocks of memory that have been dispensed by a memory allocator but that are no longer “live” (i.e., no longer being used, as determined, for example, by not being reachable from any currently referenced objects or entities). Entities that are garbage collected are typically referred to as being “dead” as opposed to being live. Garbage collection is usually handled as a background task by runtime systems rather than as an explicit task by user programs. There are some costs associated with garbage collection, as access to memory must typically be restricted and runtime systems must sometimes be halted during some portions of the garbage collection process.
Multiple runtime systems can be executed in parallel in order to increase the performance and scalability of server environments. When multiple runtime systems are executed in parallel, resource (e.g., memory) and time consumption can be reduced by sharing entities between the runtime systems. There are multiple techniques that can be used to share entities, and many different types of entities can be shared (including for example strings, constants, variables, object instances, class runtime representations, and class loaders).
A runtime system can include a cache, which is a temporary storage, in a memory, for entities. In some implementations, entities in a cache can be directly accessed by programs running in the runtime system. For example, a program can store and retrieve entities in the cache. In a cache for a Java VM, objects can be cached in the local heap.
To regulate the amount of entities in a cache, a cache can implement techniques and/or mechanisms to evict cached entities. When cached entities are evicted, runtime systems that can access the cache might not be notified of the eviction and the entity that was cached cannot be retrieved by the runtime systems. Eviction policies are used to evict entities in a cache as they provide guidance as to which cached entities should be evicted. Eviction policies can include, as examples, random eviction, round-robin eviction, and least recently used eviction.